Caption detection device, caption detection method, and pull-down signal detection apparatus

ABSTRACT

A caption detection device including a delay unit which delays a current-frame image to output a previous-frame image, a current feature detection unit which receives the current-frame image to detect a caption feature in each region, a previous feature detection unit which receives the previous-frame image from the delay unit to detect a caption feature in each region, a caption emergence region detection unit which detects a region where the caption emerges based on a temporal change between the feature in each region of the current-frame image and the feature in each region of the previous-frame image, and a caption disappearance region detection unit which detects a region where the caption disappears based on the temporal change between the feature in each region of the current-frame image from the current feature detection unit and the feature in each region of the previous-frame image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2006-324476, filed Nov. 30, 2006, theentire contents of which are incorporated herein by reference.

BACKGROUND

1. Field

One embodiment of the present invention relates to a caption detectiondevice and a caption detection method for detecting existence of acaption in a video frame and a pull-down signal detection apparatus inwhich the caption detection device and caption detection method areused.

2. Description of the Related Art

As is well known, frequently the caption for displaying a script spokenby an actor and the like is provided in a lower portion of a screen in avideo signal frame of a movie and the like. Because such pieces ofcaption information possibly have an influence in inspecting a videosignal, sometimes it is necessary to confirm whether or not the captioninformation is currently included in the video signal frame.

For example, in a caption detection method disclosed in Jpn. Pat. Appln.KOKAI Publication No. 8-212231 (Patent Document 1), an image is dividedinto plural regions, a feature amount of the caption is computed in eachregion, a determination whether or not the caption is included in eachregion is made based on the feature amount, the number of regions havingthe captions is projected in a row direction and a column direction, anda determination whether or not the caption is included in the image ismade based on the projection result.

However, in the caption detection method disclosed in Patent Document 1,because of the determination whether or not the caption information isincluded in the screen at a certain moment, there is a problem in that adistinction between video motion and the caption is hardly made indetermining how much motion exists in the video signal frame.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

A general architecture that implements the various feature of theinvention will now be described with reference to the drawings. Thedrawings and the associated descriptions are provided to illustrateembodiments of the invention and not to limit the scope of theinvention.

FIG. 1 is a block diagram showing an example of a configuration of acaption detection device according to an embodiment of the invention;

FIG. 2 is a flowchart showing an example of a determination method in acaption detection device according to an embodiment of the invention;

FIG. 3 is a flowchart showing another example of a determination methodin a caption detection device according to an embodiment of theinvention;

FIG. 4 is a block diagram showing an example of a configuration of apull-down signal detection apparatus provided with a caption detectiondevice according to an embodiment of the invention; and

FIG. 5 is a block diagram showing an example of a configuration of abroadcast receiving apparatus in which a frame interpolation unitincluding a pull-down signal detection apparatus provided with a captiondetection device according to an embodiment of the invention is used.

DETAILED DESCRIPTION

Various embodiments according to the invention will be describedhereinafter with reference to the accompanying drawings. In general,according to one embodiment of the invention, a caption detection devicecomprising:

a delay unit which delays a current-frame image to output aprevious-frame image;

a current feature detection unit which receives the current-frame imageto detect a caption feature in each region;

a previous feature detection unit which receives the previous-frameimage from the delay unit to detect a caption feature in each region;

a caption emergence region detection unit which detects a region wherethe caption emerges based on a temporal change between the feature ineach region of the current-frame image from the current-featuredetection unit and the feature in each region of the previous-frameimage from the previous feature detection unit; and

a caption disappearance region detection unit which detects a regionwhere the caption disappears based on the temporal change between thefeature in each region of the current-frame image from the currentfeature detection unit and the feature in each region of theprevious-frame image from the previous feature detection unit.

An embodiment of the invention will be described in detail withreference to the drawings.

<Caption Detection Device According to One Embodiment of the PresentInvention>

(Configuration)

A caption detection device according to an embodiment of the inventionwill be described with reference to FIG. 1. FIG. 1 is a block diagramshowing an example of a configuration of a caption detection deviceaccording to an embodiment of the invention.

Referring to FIG. 1, a caption detection device 30 includes acurrent-frame caption block detection output unit 41 to which acurrent-frame signal is supplied, a previous-frame caption blockdetection output unit 42 to which a previous-frame signal is supplied, alogic circuit 43 to which outputs of the current-frame caption blockdetection output unit 41 and previous-frame caption block detectionoutput unit 42 are supplied, a logic circuit 44, and a logic circuit 45.

The caption detection device 30 also includes an intra-screen captionemergence block counter 46 which counts a signal from the logic circuit43, a comparator 47 which performs a comparison with a caption emergencedetermination level, an intra-screen caption in-display block counter 49which counts a signal from the logic circuit 44, a comparator 50 whichperforms a comparison with a caption in-display determination level, anintra-screen caption disappearance block counter 51 which counts asignal from the logic circuit 45, a comparator 52 which performs acomparison with a caption disappearance determination level, a captionframe determination unit 48, a caption emergence block identificationunit 53 which receives a signal from the logic circuit 43, a captionin-display block identification unit 54 which receives a signal from thelogic circuit 44, and a caption disappearance block identification unit55 which receives a signal from the logic circuit 45.

The caption feature detection unit 41 includes a horizontal LPF 61 and avertical HPF 62 to which the current-frame signal (brightness signal) issupplied, a comparator 63 which performs a comparison with a verticalhigh-frequency component comparison level, an intra-small regionvertical high-frequency pixel counter 64, a comparator 65, and a logiccircuit 71. An output of the counter 64 is supplied to the comparator65, the comparator 65 compares the output of the counter 64 to avertical high-frequency block determination level, and the comparator 65supplies the comparison result to the logic circuit 71.

The caption feature detection unit 41 also includes an intra-smallregion dynamic range computing circuit 66 and a comparator 67. Thecurrent-frame signal (brightness signal) is supplied to the intra-smallregion dynamic range computing circuit 66, which computes a dynamicrange in a small region of the current-frame signal. The comparator 67compares the dynamic range in the small region to a dynamic rangecomparison level.

The caption feature detection unit 41 also includes a comparator 68, anintra-small region high-brightness pixel counter 69, and a comparator70. The current-frame signal (brightness signal) is supplied to thecomparator 68, which compares the current-frame signal to a brightnesscomparison level. The intra-small region high-brightness pixel counter69 receives the comparison result of the comparator 68 to count thenumber of high-brightness pixels in the small region. The comparator 70compares the count result to a high-brightness pixel number comparisonlevel.

Similarly, the caption feature detection unit 42 which processes theprevious-frame signal has substantially the same structure as thecaption feature detection unit 41. The caption feature detection unit 42includes a horizontal LPF 81 and a vertical HPF 82 to which theprevious-frame signal (brightness signal) is supplied, a comparator 83which performs a comparison with the vertical high-frequency componentcomparison level, an intra-small region vertical high-frequency pixelcounter 84, a comparator 85, and a logic circuit 91. An output of thecounter 84 is supplied to the comparator 85, the comparator 85 comparesthe output of the counter 84 to the vertical high-frequency blockdetermination level, and the comparator 85 supplies the comparisonresult to the logic circuit 91.

The caption feature detection unit 42 also includes an intra-smallregion dynamic range computing circuit 86 and a comparator 87. Theprevious-frame signal (brightness signal) is supplied to the intra-smallregion dynamic range computing circuit 86, which computes the dynamicrange in the small region of the previous-frame signal. The comparator87 compares the dynamic range in the small region to the dynamic rangecomparison level.

The caption feature detection unit 42 also includes a comparator 88, anintra-small region high-brightness pixel counter 89, and a comparator90. The previous-frame signal (brightness signal) is supplied to thecomparator 88, which compares the previous-frame signal to thebrightness comparison level. The intra-small region high-brightnesspixel counter 89 receives the comparison result of the comparator 88 tocount the number of high-brightness pixels in the small region.

The comparator 90 compares the count result to the high-brightness pixelnumber comparison level.

A caption detection window is supplied to the logic circuits 71 and 91.

(Operation)

The caption detection unit 30 having the above configuration detects theexistence, emergence, and disappearance of the caption by the followingoperations. FIG. 2 is a flowchart showing an example of a determinationmethod in the caption detection device of the embodiment. FIG. 3 is aflowchart showing another example of the determination method in thecaption detection device of the embodiment.

Each step of the flowcharts of FIGS. 2 and 3 can be replaced with acircuit block. Therefore, all the steps of each flowchart can beredefined as blocks.

First Processing Operation

A first processing operation performed by the caption detection unit 30of the embodiment will be described in detail. Referring to theflowchart of FIG. 2, in the caption detection unit 30, when the resultof the comparator 47 (comparison with the caption emergencedetermination level) is affirmative and, at the same time, the result ofthe comparator 50 (comparison with the caption in-display determinationlevel) is negative and, at the same time, the result of the comparator52 (comparison with the caption disappearance determination level) isnegative (YES in Step S11), the caption frame determination unit 48determines that the “caption emerges” in the frame (Step S15).

On the other hand, when the decision result of Step S11 is negative (NOin Step S11), the flow goes to Step S12. When the result of thecomparator 47 (comparison with the caption emergence determinationlevel) is negative and, at the same time, the result of the comparator50 (comparison with the caption in-display determination level) isaffirmative and, at the same time, the result of the comparator 52(comparison with the caption disappearance determination level) isnegative (YES in Step S12), the caption frame determination unit 48determines that the “caption is displayed” in the frame (Step S16).

On the other hand, when the decision result of Step S12 is negative (NOin Step S12), the flow goes to Step S13. When the result of thecomparator 47 (comparison with the caption emergence determinationlevel) is negative and, at the same time, the result of the comparator50 (comparison with the caption in-display determination level) isnegative and, at the same time, the result of the comparator 52(comparison with the caption disappearance determination level) isaffirmative (YES in Step S13), the caption frame determination unit 48determines that the “caption disappears” in the frame (Step S17).

On the other hand, when the decision result of Step S13 is negative (NOin Step S13), the caption frame determination unit 48 determines thatthe “caption does not exist” in the frame (Step S14).

Second Processing Operation

A second processing operation performed by the caption detection unit 30of the embodiment will be described in detail. Referring to theflowchart of FIG. 3, in the caption detection unit 30, when the resultof the comparator 47 (comparison with the caption emergencedetermination level) is affirmative and, at the same time, the result ofthe comparator 50 (comparison with the caption in-display determinationlevel) is negative and, at the same time, the result of the comparator52 (comparison with the caption disappearance determination level) isnegative (YES in Step S21), the caption frame determination unit 48determines whether or not at least the arbitrary number of frames inwhich the “caption does not exist” is continued to the previous frame(Step S25). When at least the arbitrary number of frames is continued(YES in Step S25), the caption frame determination unit 48 determinesthat the “caption emerges” in the frame (Step S27). When at least thearbitrary number of frames is not continued (NO in Step S25), thecaption frame determination unit 48 determines that the “caption doesnot exist” in the frame (Step S24).

On the other hand, when the decision result of Step S21 is negative (NOin Step S21), the flow goes to Step S22. When the result of thecomparator 47 (comparison with the caption emergence determinationlevel) is negative and, at the same time, the result of the comparator50 (comparison with the caption in-display determination level) isaffirmative and, at the same time, the result of the comparator 52(comparison with the caption disappearance determination level) isnegative (YES in Step S22), the caption frame determination unit 48determines that the “caption is displayed” in the frame (Step S28).

On the other hand, when the decision result of Step S22 is negative (NOin Step S22), the flow goes to Step S23. When the result of thecomparator 47 (comparison with the caption emergence determinationlevel) is negative and, at the same time, the result of the comparator50 (comparison with the caption in-display determination level) isnegative and, at the same time, the result of the comparator 52(comparison with the caption disappearance determination level) isaffirmative (YES in Step S23), the caption frame determination unit 48determines whether or not at least the arbitrary number of frames inwhich the “caption is displayed” is continued to the previous frame(Step S26). When at least the arbitrary number of frames is continued(YES in Step S26), the caption frame determination unit 48 determinesthat the “caption disappears” in the frame (Step S29). When at least thearbitrary number of frames is not continued (NO in Step S26), thecaption frame determination unit 48 determines that the “caption doesnot exist” in the frame (Step S24).

On the other hand, when the decision result of Step S23 is negative (NOin Step S23), the caption frame determination unit 48 determines thatthe “caption does not exist” in the frame (Step S24).

Thus, the caption detection device of the embodiment can determine thecaption status in each frame by at least the two methods.

<Pull-Down Signal Detection Apparatus Provided with Caption DetectionDevice According to One Embodiment of the Present Invention>

Then, a pull-down signal detection apparatus provided with the captiondetection device of the embodiment will be described. FIG. 4 is a blockdiagram showing an example of a configuration of the pull-down signaldetection apparatus in which the caption detection device of theembodiment is used.

A pull-down signal detection apparatus 119, shown in detail in FIG. 4,includes a delay unit 21, horizontal and vertical LPFs 22 and 23, aninter-frame difference absolute-value circuit 24, an intra-small regionintegration circuit 25, a dynamic block determination circuit 26, anin-screen dynamic block counter 27, a dynamic frame determinationcircuit 28, and a pull-down pattern detection circuit 29. Thecurrent-frame signal is supplied to the delay unit 21. The current-framesignal and the previous-frame signal are supplied to the horizontal andvertical LPFs 22 and 23, respectively. The inter-frame differenceabsolute-value circuit 24 detects a difference between the outputs ofthe horizontal and vertical LPFs 22 and 23. The intra-small regionintegration circuit 25 performs an integration process to thedifference. The dynamic block determination circuit 26 determineswhether or not a dynamic block exists. The in-screen dynamic blockcounter 27 counts the number of dynamic blocks. The dynamic framedetermination circuit 28 refers to a dynamic frame determination levelto determine whether or not a dynamic frame exists. The pull-downpattern detection circuit 29 detects whether or not a pull-down patternexists based on the determination result of the dynamic framedetermination circuit 28.

In the pull-down signal detection apparatus 119 having the configurationof FIG. 4, the current-frame signal and the previous-frame signal fromthe delay unit 21 are supplied to the inter-frame differenceabsolute-value circuit 24 through the horizontal and vertical LPFs 22and 23. The inter-frame difference absolute-value circuit 24 determinesthe difference between the frames in each small region, and theintra-small region integration circuit 25 performs the integration. Theintegration result is supplied to the dynamic block determinationcircuit 26, and the dynamic block determination circuit 26 performs acomparison with the dynamic block determination level to determine theblock in which the motion is observed.

At this point, the dynamic block number counting disable signalgeneration circuit 31 performs a process by receiving a caption framedetermination output, a caption emergence block flag, a captionemergence block number, a caption disappearance block flag, and acaption disappearance block number from the output of the captiondetection device 31. In detecting the pull-down signal, the intra-screendynamic block counter 27 does not count the emergence and disappearanceof the caption, which are not the original motion of the screen, butcounts only the motion of the real image.

Therefore, even if the caption emerges and disappears, because theintra-screen dynamic block counter 27 does not count the emergence anddisappearance of the caption, the dynamic frame determination circuit 28can make the comparison with the dynamic frame determination level tocorrectly determine whether the real motion exists in a given videosignal. Accordingly, only the determination of the correct pull-downvideo signal can be made and the subsequent pull-down pattern detectionunit 29 can accurately detect, e.g., a 2-3 pull-down signal and a 2-2pull-down signal.

<Broadcast Receiving Apparatus with Frame Interpolation ApparatusIncluding Pull-Down Signal Detection Apparatus Provided with CaptionDetection Device According to One Embodiment of the Present Invention>

FIG. 5 is a block diagram showing an example of a configuration of abroadcast receiving apparatus in which the frame interpolation apparatusincluding the pull-down signal detection apparatus is used. In thepull-down signal detection apparatus, the caption detection device ofthe embodiment is used.

In a broadcast receiving apparatus 10, the pull-down signal detectionapparatus provided with the caption detection device of the embodimentis used. Referring to FIG. 5, the broadcast receiving apparatus 10includes a tuner 111, an MPEG decoder 114, an IP conversion unit andscaling unit 115, an image quality correction unit 116, a frameinterpolation apparatus 117, and a display 113. The tuner 111 outputsaudio and video signals by receiving an analog surface wave, an analogsatellite wave, a digital surface wave, or a digital satellite wave. TheMPEG decoder 114 performs an MPEG decoding process to the audio andvideo signals. The IP conversion unit and scaling unit 115 performs IPconversion and a scaling process to the decoded signals. The imagequality correction unit 116 performs an image quality correctionprocess. The frame interpolation unit 117 performs a frame interpolationprocess to a given 60-Hz (50 Hz in Europe) video signal to output a120-Hz (100 Hz in Europe) video signal which is a double speed. Thedisplay 113 displays a video according to the video signal to which theinterpolation process is performed.

When the pull-down signal detection apparatus 119 including the captiondetection device 30 of the embodiment is applied to the frameinterpolation apparatus 117 of the broadcast receiving apparatus 10, thenumber of intra-screen dynamic block counts can be obtained based on thecorrect determination without influence of the presence or absence ofthe caption.

In the conventional technique, the existence of the caption in the videoscreen causes such troubles that the signal is mistakenly recognized asthe pull-down signal or the pull-down signal is mistakenly recognized asthe usual video signal, whereby a malfunction is generated by notselecting the original interpolation processing units 118 and 118-2.

Those skilled in the art can make the invention by variousabove-described embodiments, and it is further understood by thoseskilled in the art that various changes and modifications can easily bemade, and the invention can be applied to various embodiments withoutinventive ability. Therefore, the present invention covers a wide rangeconsistent with the disclosed principle and the novel characteristicfeatures, and is not limited to the embodiments described above.

While certain embodiments of the inventions have been described, theseembodiments have been presented by way of example only, and are notintended to limit the scope of the inventions. Indeed, the novel methodsand systems described herein may be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods and systems described herein may be made withoutdeparting from the spirit of the inventions. The accompanying claims andtheir equivalents are intended to cover such forms or modifications aswould fall within the scope and spirit of the inventions.

1. A pull-down signal detection apparatus comprising: a differencecircuit which detects a difference between a frame of a current-frameimage and a frame of a previous-frame image delayed from thecurrent-frame image; a determination unit which determines a block inwhich motion is observed based on the difference; a caption detectionunit including: a current feature detection unit which receives thecurrent-frame image to detect a feature in each of regions obtained bydividing the current-frame image, a previous feature detection unitwhich receives the delayed previous-frame image to detect a feature ineach of regions obtained by dividing the previous-frame image, a captionemergence region detection unit which detects a region where the captionemerges based on a temporal change representing how the caption featurechanges with time between a predetermined region of the previous-frameimage and a predetermined region of the current-frame image, thetemporal change being derived from the feature in each of the regions ofthe current-frame image from the current-feature detection unit and thefeature in each of the regions of the previous-frame image from theprevious feature detection unit, and a caption disappearance regiondetection unit which detects a region where the caption disappears basedon the temporal change representing how the caption feature changes withtime between the predetermined region of the previous-frame image andthe predetermined region of the current-frame image, the temporal changebeing derived from the feature in each of the regions of thecurrent-frame image from the current feature detection unit and thefeature in each of the regions of the previous-frame image from theprevious feature detection unit; a counter which performs counting bysubtracting blocks in which the caption emerges and disappears detectedby the caption detection unit from blocks in which motion is observeddetermined by the dynamic block determination unit; a determination unitwhich determines a dynamic frame according to the number of counts ofthe counter; and a detection unit which outputs a signal indicating thata pull-down signal is detected, when the dynamic frame determined by thedetermination unit is a pull-down pattern.
 2. The pull-down signaldetection apparatus according to claim 1, further comprising: a tunerwhich receives and demodulates a broadcast signal to output a frameimage signal, and supplies the frame image signal to the differencecircuit; a frame interpolation unit which performs a frame interpolationprocess to the frame image signal from the tuner based on presence orabsence of the pull-down signal detected by the detection unit; and adisplay unit which receives a video signal to display a video on ascreen according to the video signal, the video signal beinginterpolated in the frame by the frame interpolation unit.
 3. Apull-down signal detection method comprising: detecting a differencebetween a frame of a current-frame image and a frame of a previous-frameimage delayed from the current-frame image; determining a block in whichmotion is observed based on the detected difference; receiving thecurrent-frame image to detect a feature in each of regions obtained bydividing the current-frame image, receiving the delayed previous-frameimage to detect a feature in each of regions obtained by dividing theprevious-frame image, detecting a region where the caption emerges basedon a temporal change representing how the caption feature changes withtime between a predetermined region of the previous-frame image and apredetermined region of the current-frame image, the temporal changebeing derived from the feature in each of the regions of thecurrent-frame image from the current-feature detection unit and thefeature in each of the regions of the previous-frame image from theprevious feature detection unit; detecting a region where the captiondisappears based on the temporal change representing how the captionfeature changes with time between the predetermined region of theprevious-frame image and the predetermined region of the current-frameimage, the temporal change being derived from the feature in each of theregions of the current-frame image from the current feature detectionunit and the feature in each of the regions of the previous-frame imagefrom the previous feature detection unit; counting by subtracting blocksin which the caption emerges and disappears detected by the captiondetection unit from blocks in which motion is observed determined by thedynamic block determination unit; determining a dynamic frame accordingto the number of counts of the counter; and outputting a signalindicating that a pull-down signal is detected, when the dynamic framedetermined by the determination unit is a pull-down pattern.